Information management apparatus, image recording apparatus, image pickup apparatus, image management system, image analysis information management method, and program

ABSTRACT

An image management apparatus may include an input image setting information acquiring unit configured to, when image analysis information on an input image is set, acquire setting information as input image setting information, an available setting information acquiring unit configured to acquire setting information as available setting information, an update necessity determining unit configured to determine whether or not an update of the image analysis information is necessary, on the basis of a difference between the input image setting information and the available setting information, and an image analysis information setting unit configured to, when it is determined that an update of the image analysis information is necessary, perform image analysis on the input image using the second image analysis processing unit so as to set new image analysis information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from Japanese Patent Application No. JP2006-189894 filed in the Japanese Patent Office on Jul. 10, 2006, theentire content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to information management systems. Inparticular, the present invention relates to an image management systemfor managing an image and analysis information on the image, an imagemanagement apparatus, an image recording apparatus, an image pickupapparatus, a processing method for the above system and apparatuses, anda program for causing a computer to execute the method.

2. Description of the Related Art

Recently, with the increasingly widespread use of image pickupapparatuses such as digital still cameras, the number of images that canbe recorded in such apparatuses has increased. This increase in therecording capacity makes it difficult to manage and view recordedimages. Thus, techniques have been proposed for facilitating imagemanagement. For example, additional information is set for each imageand the set additional information is used as a search condition forsearching for the image.

According to an image management device disclosed in Japanese UnexaminedPatent Application Publication No. 2005-354134, image pickup informationsuch as an image pickup date and a keyword is set as additionalinformation for each image, and then a unified keyword can be set.

SUMMARY OF THE INVENTION

When images are managed by setting additional information, as in theabove case, conditions under which the additional information is set maycause a problem. For example, in a case where the content of an image isanalyzed using an image analysis engine and information obtained as aresult of the analysis is set as additional information for the image,the image analysis engine may be updated or improved (version up) afterthe additional information is set. This adoption of a later version ofone image analysis engine may result in the content the additionalinformation being changed.

Even when the version of the image analysis engine is not changed, theprecision of analysis may be limited due to the operating conditions ofthe image analysis engine. For example, in a body of a digital stillcamera, the analysis precision of an image analysis engine isintentionally restricted in consideration of battery consumption andperformance limitations of an operating processor. In addition, imageanalysis with high precision may be realized by connecting the digitalstill camera to another device such as a server.

However, the time at which the digital still camera is connected to theserver depends on a user. Thus, it is difficult to determine the type ofimage analysis engine by which additional information stored in thedigital still camera is created, at a time when the digital still camerais connected to the server.

Thus, there is a need for a technique in which when an image andanalysis information on the image are managed, the image analysisinformation is reset as necessary on the basis of setting informationregarding an image analysis engine that has performed image analysiscorresponding to the set image analysis information.

The present invention has been made in view of the above circumstance.According to an embodiment of the present invention, an image recordingapparatus may include image analyzing means configured to executepredetermined image analysis processing on an input image so as toextract image analysis information, image analysis processing typeacquiring means configured to acquire a type of the executed imageanalysis processing, and recording controlling means configured torecord the image analysis information and the type of the image analysisprocessing in a recording medium so as to be associated with the inputimage. With this image recording apparatus, image analysis informationand a type of image analysis information can be recorded so as to beassociated with an input image. An example of a type of image analysisinformation is analysis engine information 93.

According to an embodiment of the present invention, an image managementapparatus may include input image setting information acquiring meansconfigured to, when image analysis information on an input image is set,acquire setting information as input image setting information, thesetting information relating to a first image analysis processing unitwhich has set the image analysis information, available settinginformation acquiring means configured to acquire setting information asavailable setting information, the setting information relating to asecond image analysis processing unit which is available, updatenecessity determining means configured to determine whether or not anupdate of the image analysis information is necessary, on the basis of adifference between the input image setting information and the availablesetting information, and image analysis information setting meansconfigured to, when it is determined that an update of the imageanalysis information is necessary, perform image analysis on the inputimage using the second image analysis processing unit so as to set newimage analysis information. This image management apparatus may enablecontrol as to whether or not new image analysis information is set onthe basis of input image setting information and available settinginformation.

In this image management apparatus, the input image setting informationmay contain information on a version of the first image analysisprocessing unit, the available setting information may containinformation on a version of the second image analysis processing unit,and the update necessity determining means may determine that an updateof image analysis information is necessary if the version informationcontained in the available setting information indicates a later versionthan the version information contained in the input image settinginformation. This arrangement may enable control as to whether or notnew image analysis information is set on the basis of a differencebetween the version information of input image setting information andthe version information of available setting information.

In addition, in the image management apparatus, the input image settinginformation may contain information on a parameter of the first imageanalysis processing unit, the available setting information may containinformation on a parameter of the second image analysis processing unit,and the update necessity determining means may determine that an updateof image analysis information is necessary, if the version informationcontained in the available setting information and the versioninformation contained in the input image setting information indicatethe same version and the parameter information contained in theavailable setting information may indicate a higher analysis precisionthan the parameter information contained in the input image settinginformation. This arrangement may enable control as to whether or notnew image analysis information is set on the basis of a differencebetween the parameter information of input image setting information andthe parameter information of available setting information.

Further, in the image management apparatus, the image analysisinformation on the input image may include analysis information on aface image contained in the input image and analysis informationrelating to a degree of similarity between the input image and areference image.

Moreover, in the image management apparatus, the determination as to thenecessity of an update of the image analysis information may betriggered by receipt of the input image or by an update of the secondimage analysis processing unit.

According to an embodiment of the present invention, an image recordingapparatus may include image recording means configured to record aninput image, input image setting information acquiring means configuredto, when image analysis information on the input image is set, acquiresetting information as input image setting information, the settinginformation relating to a first image analysis processing unit which hasset the image analysis information, available setting informationacquiring means configured to acquire setting information as availablesetting information, the setting information relating to a second imageanalysis processing unit which is available, update necessitydetermining means configured to determine whether or not an update ofthe image analysis information is necessary, on the basis of adifference between the input image setting information and the availablesetting information, and image analysis information setting meansconfigured to, when it is determined that an update of the imageanalysis information is necessary, perform image analysis on the inputimage using the second image analysis processing unit so as to set newimage analysis information. This image recording apparatus may enablecontrol as to whether or not new image analysis information is set onthe basis of a difference between input image setting informationcorresponding to an input image and available setting information.

According to an embodiment of the present invention, an image pickupapparatus may include image pickup means configured to pick up an imageof a subject as an input image, input image setting informationacquiring means configured to, when image analysis information on theinput image is set, acquire setting information as input image settinginformation, the setting information relating to a first image analysisprocessing unit which has set the image analysis information, availablesetting information acquiring means configured to acquire settinginformation as available setting information, the setting informationrelating to a second image analysis processing unit which is available,update necessity determining means configured to determine whether ornot an update of the image analysis information is necessary, on thebasis of a difference between the input image setting information andthe available setting information, and image analysis informationsetting means configured to, when it is determined that an update of theimage analysis information is necessary, perform image analysis on theinput image using the second image analysis processing unit so as to setnew image analysis information. This image pickup apparatus may enablecontrol as to whether or not new image analysis information is set onthe basis of a difference between input image setting informationcorresponding to a picked up input image and available settinginformation.

According to an embodiment of the present invention, an image managementsystem may have an image recording apparatus recording an input imageand an image management apparatus managing image analysis information onthe input image. The image management apparatus may include input imagesetting information acquiring means configured to receive the inputimage from the image recording apparatus, and when image analysisinformation on the input image is set, acquire setting information asinput image setting information, the setting information relating to afirst image analysis processing unit which has set the image analysisinformation, available setting information acquiring means configured toacquire setting information as available setting information, thesetting information relating to a second image analysis processing unitwhich is available, update necessity determining means configured todetermine whether or not an update of the image analysis information isnecessary, on the basis of a difference between the input image settinginformation and the available setting information, and image analysisinformation setting means configured to, when it is determined that anupdate of the image analysis information is necessary, perform imageanalysis on the input image using the second image analysis processingunit so as to set new image analysis information, and image analysisinformation supplying means configured to supply the new image analysisinformation to the image recording apparatus. This image managementsystem may enable control as to whether or not new image analysisinformation is set and supplied from the image management apparatus tothe image recording apparatus, on the basis of a difference betweeninput image setting information and available setting information.

According to the present invention, when an image and analysisinformation on the image are managed, the image analysis information canadvantageously be reset as necessary on the basis of setting informationrelating to an image analysis engine that has performed image analysiscorresponding to the set image analysis information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an entire configuration of an image management systemaccording to an embodiment of the present invention;

FIG. 2 illustrates an example of image analysis information according toan embodiment of the present invention;

FIG. 3 illustrates an example of analysis engine information accordingto an embodiment of the present invention;

FIG. 4 illustrates a functional configuration of an image managementapparatus according to an embodiment of the present invention;

FIG. 5 illustrates a first example of a case in which an update of imageanalysis information is necessary, according to an embodiment of thepresent invention;

FIG. 6 illustrates a second example of a case in which an update ofimage analysis information is necessary, according to an embodiment ofthe present invention;

FIG. 7 illustrates a third example of a case where an update of imageanalysis information is necessary, according to an embodiment of thepresent invention;

FIG. 8A to 8C illustrate examples of relationships between image dataand image analysis information, according to an embodiment of thepresent invention;

FIGS. 9A and 9B are schematic views of an operation part of a digitalstill camera according to an embodiment of the present invention;

FIG. 10 illustrates a processing procedure performed by an imagemanagement apparatus according to an embodiment of the presentinvention; and

FIG. 11 illustrates a processing procedure of image analysis informationupdate processing according to an embodiment of the present invention.

DETAILED DESCRIPTION

In the following, the preferred embodiments of the present inventionwill be described in detail with reference to the accompanying drawings.

FIG. 1 illustrates an example of an entire configuration of an imagemanagement system according to an embodiment of the present invention.The system includes a server 10, an image database 19, a digital stillcamera 21, and a mobile phone 22.

The server 10 stores image data 91 input from the digital still camera21 or the mobile phone 22 in the image database 19. The image database19 stores the image data 91 input from the digital still camera 21 orthe mobile phone 22. The image database 19 manages the image data 91together with analysis information on an image contained in the imagedata 91. Each piece of the image data 91, a corresponding thumbnailimage, and corresponding image analysis information are provided with acommon content identifier so as to be linked to each other.

The server 10 has an image analysis engine for performing image analysison the input image data 91 as necessary. This image analysis engine canbe implemented as a program operating on a computer or as a hardwaredevice.

When the image analysis engine is implemented as a program, the programcan be updated or modified via a network 50. Through this update ormodification, the version of the image analysis engine is changed. Suchprocessing of changing the version of an image analysis engine isreferred to as version-up. In addition, when the image analysis engineis implemented as a hardware device, version up is executed by thereplacement of a circuit board, for example.

The content of image analysis performed by the image analysis engineincludes, for example, analysis of an image of a human face contained inan image and analysis of a degree of similarity between the image and areference image. A result of the image analysis is output as imageanalysis information 92. In addition, analysis engine information 93 isalso output as information on the image analysis engine that hasgenerated the analysis result. The analysis engine information 93 caninclude version information 94, which is information on a version of theimage analysis engine, and parameter information 95, which isinformation on a parameter provided to the image analysis engine.

The image analysis information 92 and the analysis engine information 93generated by the image analysis engine are managed by the image database19 together with the image data 91, and then written back to the digitalstill camera 21 or the mobile phone 22 that has input the image data 91.

The digital still camera 21 and the mobile phone 22 are examples ofimage recording apparatuses for recording the image data 91. Forexample, the digital still camera 21 has an image pickup unit forpicking up an image of a subject as an input image and stores the imagedata 91 of the picked up image in an internal memory, an externalrecording medium or the like. If the mobile phone 22 has an image pickupfunction, the mobile phone 22 can similarly record images. The imagedata 91 is not limited to a picked up image. For example, an imageacquired from an external unit using a communication function can alsobe stored as the image data 91 in the internal memory, the externalrecording medium, or the like.

The digital still camera 21 and the mobile phone 22 can also containimage analysis engines, which permit image analysis of the recordedimage data 91. However, in general, the battery consumption andprocessor performance of such a portable-type device are often limited,and thus it is difficult to perform precise image analysis using asingle portable-type device. Thus, when the digital still camera 21 orthe like contains an image analysis engine, image analysis is firstperformed in a simple manner using the image analysis engine, and thendetailed image analysis can be performed after the image data 91 istransferred to the server 10. With this arrangement, image analysis withhigh precision can be executed using the server 10 having high operatingperformance. In addition, a result of the image analysis is rewrittenback to the digital still camera 21 or the like, so that the analysisresult with high precision can be stored in the digital still camera 21or the like.

FIG. 2 illustrates an example of the image analysis information 92according to an embodiment of the present invention. The image analysisinformation 92 can be provided as metadata for image data and can bedescribed in a text format or a binary format. In this example, theimage analysis information 92 is assumed to be described in an XML(extensible Mark-up Language) format.

A <photo> tag pair indicates image analysis information. In an XMLformat, a description placed between a start tag and a corresponding endtag represents the content of a tag pair. For example, a start tag ofthe <photo> tag is expressed as “<photo>” and a corresponding end tag isexpressed as “</photo>”.

A <guid> tag pair indicates a content identifier of a correspondingimage. The content identifier is composed of 128 bits and providedcommonly to the image (original image) and a reduced image of the image(thumbnail image).

A <FullImgPath> tag pair indicates a location of a file containing theimage data of the original image by a file path and a file name. A<CacheImgPath> tag pair indicates a location of a file containing imagedata of the reduced image by a file path and a file name.

A <TimeStamp> tag pair indicates date and time when the image wascaptured. The example of FIG. 2 indicates that the image was captured at6:52:32, on Mar. 31 2003.

A <FaceInfo> tag pair indicates information on an image of a face of ahuman or the like (hereinafter referred to as a face image) included inthe image corresponding to the image data. Such information is generatedby a face detection engine in the image analysis engine. In accordancewith the number of face images included in the image, information oneach face image is described as follows.

A <TotalFace> tag pair indicates the total number of face imagescontained in the image data. The example of FIG. 2 indicates that thenumber of face images contained in the image data is “1”.

A <FaceEntry> tag pair indicates specific information on each faceimage. Since only one face images is contained in the image data in thisexample, only one <FaceEntry> tag pair is provided.

An <x> tag pair indicates a position in a face image in a horizontaldirection (X direction) of a face image. The example of FIG. 2 indicatesthat the position of the right end of the face image in the horizontaldirection is “0.328767”, where the left end of the entire imagecorresponding to the image data is represented as “0.0” and the right isrepresented as “1.0”. A <y> tag pair indicates a position in the faceimage in a vertical direction (Y direction). The example of FIG. 2indicates that the position of the upper end of the face image in thevertical direction is “0.204082”, where the upper end of the entireimage is represented as “0.0” and the lower end of the image isrepresented as “1.0”. Specifically, a normalized value corresponding toa position in the face image with respect to the horizontal direction iswritten between the <x> tag and the </x> tag, and a normalized valuecorresponding to a position in the face image with respect to thevertical direction is written between the <y> tag and the </y> tag.

A <width> tag pair indicates a width of a face image (size in thehorizontal direction). In this example, the width of the face image is“0.408163”, where the width of the entire image corresponding to theimage data is represented as “1.0”. A <height> tag pair indicates aheight of the face image (size in the vertical direction). In thisexample, the height of the face image is “0.273973”, where the height ofthe entire image is represented as “1.0”. Specifically, a normalizedvalue corresponding to the width of the face image is written betweenthe <width> tag and the </width> tag, and a normalized valuecorresponding to the height of the face image is written between the<height> tag and the </height> tag.

A <roll> tag pair indicates a roll angle of a face image (an angle ofrotation around a longitudinal axis (x-axis) indicative of a position ofthe face image in the superior/inferior direction). In this example, itis indicated that the roll angle of the face image is “0.000000”.

A <pitch> tag indicates a pitch angle of a face image (an angle ofrotation around a transverse axis (y-axis) indicative of a position ofthe face image in the right/left direction). In this example, it isindicated that the pitch angle of the face image is “0.000000”.

A <yaw> tag pair indicates a yaw angle of a face image (an angle ofrotation around a normal axis (z-axis) indicative of a position of theface image in the anteroposterior direction). In this example, it isindicated that the yaw angle of the face image is “0.000000”.

A <SimilarityInfo> tag pair indicates information on a feature quantityrepresenting a degree of similarity between images. This information isgenerated by an image similarity analysis engine in an image analysisengine. In this example, feature quantities for calculating a degree ofassociation indicative of a degree in which the image is associated witha predetermined color name and a degree of similarity to another imagein terms of color or frequency components are described as follows.

A <ColorInfo> tag pair indicates a degree of association indicative of adegree in which the image data is associated with a predetermined colorname. In this example, degrees of association in terms of six colors aredescribed. A <ColorWhite> tag pair, a <ColorBlack> tag pair, a<ColorRed>tag pair, a <ColorYellow> tag pair, and a <ColorGreen> tagpair indicate degrees in which the image data is associated with colorsof white, black, red, yellow, green, and blue, respectively. As shown inFIG. 2, the degree of association in terms of green is “12” percent andthe degree of association in terms of each of the other colors is “0”percent.

A <VectorInfo> tag pair indicates a feature quantity representing adegree of similarity between images. In this example, three <VectorInfo>tag pairs are provided, and a <method> tag pair and a <vector> tag pairare written between each of the <VectorInfo> tag and </VectorInfo> tags.The <method> tag pair indicates a method for obtaining a degree ofsimilarity in terms of a specific feature. The <vector> tag pairindicates a vector representing a corresponding feature quantity. In theexample of FIG. 2, a feature quantity for color based on a Gaussianmodel, a feature quantity for pattern, and a feature quantity for shapeare described.

FIG. 3 illustrates an example of the analysis engine information 93according to an embodiment of the present invention. This analysisengine information 93 may be provided as metadata for image data andwritten either in a text format or in a binary format. In this example,it is assumed that the analysis engine information 93 is written in anXML format.

A <metainfo> tag pair indicates analysis engine information. A <guid>tag pair indicates a content identifier of a corresponding image,similarly to the example of FIG. 2.

A <metaSFaceEnableFlag> tag pair indicates whether or not a face imageis detected in the image content. In this example, “1”, indicating thata face image is detected in the image content, is written.

A <metaSIBSEnableFlag> tag pair indicates whether or not similar imageanalysis has been performed on the image content. In this example, “1”,indicating that similar image analysis has been performed on the imagecontent, is written.

A <metaSFaceCfgInfo> tag pair indicates information related to facedetection including version information and parameter informationregarding a face detection engine, as described below.

A <verMajor> tag pair indicates major version information. The exampleof FIG. 3 indicates that the major version is “SFACE 1.2.0.06013100”.The number placed subsequent to “SFACE” indicative of a face detectionengine is a major version number in which a large number indicates alater version.

A <verMinor> tag pair indicates minor version information. The exampleof FIG. 3 indicates that the minor version is “IFLCore 2.4.01”. Thenumber placed subsequent to “IFLCore” indicative of a type of a facedetection engine is a minor version number in which a larger numberindicates a later version.

In the above example, the version information of the face detectionengine includes the major version information and the minor versioninformation. However, unified version information can be employed as theversion information of the face detection engine.

Items written between the <FaceEntry> tags described with reference toFIG. 2 may depend on the version of a face detection engine. Forexample, when the version of the face detecting engine is old, the tagsindicating the orientation of a face image such as the <roll>, <pitch>,and <yaw> tags may not be included. On the other hand, when the versionof the face detecting engine is new, a tag pair indicative of a faceexpression may be included.

Referring back to FIG. 3, a <cfgParam> tag pair indicates parameterinformation. In this example, a parameter provided to the face detectingengine is “{false,false,15,30,10,10,160,120,−0.8869f,−0.8869f,“MVIEWI_P12/V5”,450,340}”. The two numbers provided at the end of theparameter (i.e., 450 and 340) indicate the resolution in which facedetection is performed as a size in the horizontal and verticaldirections. The example of FIG. 3 indicates that the resolution of facedetection is 450 horizontal pixels×340 vertical pixels. That is, theprecision of face detection increases as the size increases anddecreases as the size decreases.

For example, when the resolution of face detection is 315 horizontalpixels×236 vertical pixels, the level of detection precision is nothigh. However, in this case, only a small amount of computation isnecessary for the face detection. On the other hand, when the resolutionof the face detection is 550 horizontal pixels×412 vertical pixels, alarge amount of computation is necessary whereas the level of detectionprecision is high. In addition, when the resolution of the facedetection is 450 horizontal pixels×340 vertical pixels, face detectionwith an intermediate level of precision and an intermediate amount ofcomputation can be achieved.

The total number of face images written between the <TotalFace> tagsdescribed with reference to FIG. 2 may depend on the parameter of theface detection engine. For example, an increase in detection precisionindicated by the parameter may indicate an increase in the total numberof face images that can be detected.

A <metaSIBSCfgInfo> tag pair indicates information relating to imagesimilarity analysis which includes version information and parameterinformation of an image similarity analysis engine, as described below.

A <verMajor> tag pair indicates major version information. The exampleof FIG. 3 indicates that the major version is “SIBS CORE 0.6.05112400”.Specifically, the number placed subsequent to “SIBS CORE” indicative ofan image similarity analysis engine is a major version number in which alarge number indicates a later version.

A <verMinor> tag pair indicates minor version information. The exampleof FIG. 3 indicates that the minor version is “MDS 1.0.0”. Specifically,the number placed subsequent to “MDS” indicative of a type of imagesimilarity analysis engine is a minor version number in which a largernumber indicates a later version.

In the above example, the version information of the image similarityanalysis engine includes the major version information and the minorversion information. However, unified version information can beemployed as the version information of the image similarity analysisengine.

The number of colors written between the <ColorInfo>tags may depend onthe version of the image similarity analysis engine. For example, if theversion is updated, the number of colors written between the <ColorInfo>tags may be increased.

A <cfgParam> tag pair indicates parameter information. The example ofFIG. 3 indicates that a parameter provided to the image similarityanalysis engine is “{30,1,2,20,10,20,2,160,120}”. The two numbers placedat the end of the parameter (i.e., 160 and 120) indicate resolution inwhich similar image analysis is performed as a size in the horizontaland vertical directions (in this example, the resolution of facedetection is 160 horizontal pixels×120 vertical pixels). That is, theprecision of face detection increases as the size increases anddecreases as the size decreases.

For example, when the resolution of face detection is 160 horizontalpixels×120 vertical pixels, the level of detection precision is nothigh. However, in this case, only a small amount of computation isnecessary for the face detection. On the other hand, when the resolutionof the face detection is 640 horizontal pixels×480 vertical pixels, alarge amount of computation is necessary whereas the level of detectionprecision is high. In addition, when the resolution of the facedetection is 320 horizontal pixels×240 vertical pixels, face detectionwith an intermediate level of precision and an intermediate amount ofcomputation can be achieved.

The accuracy of a degree of association indicated by the <ColorInfo>tags and the accuracy of a feature quantity indicated by the<VectorInfo> tags which are described with reference to FIG. 2 maydepend on the parameter of the image similarity analysis engine. Forexample, an increase in a parameter indicating detection precision(precision parameter) may increase the accuracy of the degree ofassociation and the feature quantity.

FIG. 4 illustrates a functional configuration of an image managementapparatus according to an embodiment of the present invention. Thisimage management apparatus receives an input image file 90 and outputs asaved image file 190. As described above, the input image file 90contains the image data 91, the image analysis information 92, and theanalysis engine information 93, as described above. Likewise, the savedimage file 190 contains image data 191, image analysis information 192,and analysis engine information 193. The saved image file 190 is storedin the image database 19.

The image management apparatus has a trigger detecting unit 110, asetting presence determining unit 120, a image analysis engine 130, aninput information acquiring unit 140, an engine information acquiringunit 150, an update necessity determining unit 160, a setting necessitydetermining unit 170, a data setting unit 181, an analysis informationsetting unit 182, an engine information setting unit 183, and a writeback unit 199.

The trigger detecting unit 110 detects a trigger for setting imageanalysis information. Types of trigger will be described below.

The image analysis engine 130 performs image analyses on the image data91. The content of the image analysis performed by the image analysisengine 130 includes, for example, analysis of a face image contained inthe image and analysis of a degree of similarity between the image and areference image. The image analysis engine 130 stores analysis engineinformation 132 as information on an engine body 131. This analysisengine information 132 contains version information and parameterinformation on the engine body 131.

The setting presence determining unit 120 determines whether or not theimage analysis information 92 is set in the input image file 90. Aresult of the determination is supplied to the setting necessitydetermining unit 170.

When the input image file 90 contains the analysis engine information93, the input information acquiring unit 140 acquires the analysisengine information 93. The analysis engine information 93 acquired bythe input information acquiring unit 140 is supplied to the updatenecessity determining unit 160.

The engine information acquiring unit 150 acquires the analysis engineinformation 132 of the image analysis engine 130. The analysis engineinformation 132 acquired by the engine information acquiring unit 150 issupplied to the update necessity determining unit 160.

The update necessity determining unit 160 determines whether or not anupdate of image analysis information is necessary, on the basis of adifference between the analysis engine information 93 of the input imagefile 90 acquired by the input information acquiring unit 140 and theanalysis engine information 132 of the image analysis engine 130acquired by the engine information acquiring unit 150. A result of thedetermination performed by the update necessity determining unit 160 issupplied to the setting necessity determining unit 170.

When the version indicated in the analysis engine information 132 of theimage analysis engine 130 is later than the version indicated in theanalysis engine information 93 of the input image file 90, the updatenecessity determining unit 160 determines that an update of imageanalysis information is necessary. Specifically, if the version of imageanalysis engine 130 that is available is later than an image analysisengine that has generated the image analysis information 92, the updatenecessity determining unit 160 determines that the image analysisinformation 92 needs to be reset using the image analysis engine 130.

In addition, when the version indicated in the analysis engineinformation 132 of the image analysis engine 130 and the versionindicated in the analysis engine information 93 of the input image file90 are the same, and the precision parameter contained in the analysisengine information 132 of the image analysis engine 130 is higher thanthe precision parameter contained in the analysis engine information 93of the input image file 90, the update necessity determining unit 160also determines that an update of image analysis information isnecessary. Specifically, when the image analysis engine 130 that isavailable is capable of performing analysis with higher precision thanthe image analysis engine corresponding to the analysis engineinformation 93, the update necessity determining unit 160 determines theimage analysis information 92 needs to be reset even if the versions ofboth the image analysis engine are the same.

When the image analysis engine includes a plurality of modules as in theabove example, (i.e., the face detecting engine and the image similarityanalysis engine in the example of FIG. 3), image analysis can beperformed only for a desired module.

The setting necessity determining unit 170 determines whether or not theimage analysis information 192 is set in accordance with the image data191. The setting necessity determining unit 170 determines that theimage analysis information 192 needs to be set, when the settingpresence determining unit 120 determines that the image analysisinformation 92 is not set in the input image file 90 or when the updatenecessity determining unit 160 determines that an update of imageanalysis information is necessary. A result of the determination of thesetting necessity determining unit 170 is supplied to the image analysisengine 130, the data setting unit 181, the analysis information settingunit 182, and the engine information setting unit 183.

If the setting necessity determining unit 170 determines that the imageanalysis information 192 needs to be set, the image analysis engine 130performs image analysis on the image data 91. Then, the data settingunit 181 saves the image data 91 as the image data 191, the result ofthe analysis performed by the image analysis engine 130 is set as theimage analysis information 192, and the analysis engine information 132of the image analysis engine 130 is set as the analysis engineinformation 193.

On the other hand, when the setting necessity determining unit 170determines that the image analysis information 192 does not need to beset, the image analysis engine 130 does not perform image analysis.Then, the data setting unit 181 saves the image data 91 as the imagedata 191. At this time, if the image analysis information 92 has alreadybeen set, the image analysis information 92 is saved as the imageanalysis information 192, and the analysis engine information 93 issaved as the analysis engine information 193.

The write back unit 199 writes output of the image analysis engine 130back to a source of the input image file 90. When the image analysisinformation 192 is set by the image analysis engine 130, the write backunit 199 writes the image analysis information 192 and the correspondinganalysis engine information 132 (analysis engine information 193) backto the source as necessary. It is possible to determine beforehandwhether or not such write back is necessary.

Now, cases where an update of image analysis information is necessaryaccording to an embodiment of the present invention will be described.

FIG. 5 illustrates a first example of the case in which an update ofimage analysis information is necessary according to an embodiment ofthe present invention. In this example, the digital still camera 21serves as an image recording apparatus, and the server 10 serves as animage management apparatus. The server 10 has an image analysis engine.It is assumed that the digital still camera 21 does not have an imageanalysis engine or does have an image analysis engine of an old versionor an image analysis engine with low image analysis precision.

Here, a case is assumed in which the digital still camera 21 isconnected to the server 10 so that images recorded in the digital stillcamera 21 are backed up. Upon the connection to the digital still camera21, the server 10 performs setting of image analysis information forimage data. Then, the image analysis information is written back to thedigital still camera 21. As a result, the image analysis informationprocessed by the image analysis engine of the newest version at thistime is reflected on the digital still camera 21. When the write back iscompleted, the digital still camera 21 and the server 10 aredisconnected.

It is further assumed that version-up of the image analysis engine isexecuted in the server 10 after the write back described above. In thiscase, when the digital still camera 21 is connected to the server 10again, the server 10 performs setting of image analysis information forthe image data again, since the version of the image analysis engine ofthe server 10 is later than the version corresponding to image analysisinformation stored in the digital still camera 21 (or the precisionparameter of the image analysis engine of the server 10 is higher thanthe precision parameter corresponding to the image analysis informationof the digital still camera 21). Then, new image analysis informationobtained as a result of the reset of image analysis information iswritten back to the digital still camera 21.

In this example, the trigger detecting unit 110 detects the connectionbetween the digital still camera 21 and the server 10. This connectionserves as a trigger for causing the necessity determining unit 170 todetermine whether or not setting of image analysis information isnecessary.

In the above example, the case is described where the version of theimage analysis engine of the server 10 is updated. However, there mayalso be a case where the digital still camera 21 is connected to anotherserver after being disconnected from the server 10, and the version ofan image analysis engine of the other server is later than that of theserver 10 (or the precision parameter provided to the other server ishigher than that provided to the server 10). Also in such a case, theprocedure of the first example described above is similarly executed.

FIG. 6 illustrates a second example of the case in which an update ofimage analysis information is necessary according to an embodiment ofthe present invention. In this example, it is assumed that a computer 31has an image analysis engine and functions of both an image recordingapparatus and an image management apparatus.

The computer 31 has an image recording unit for recording input imagedata. After recording image data, the computer 31 sets image analysisinformation on the image data using the image analysis engine. Here, itis assumed that a version-up of the image analysis engine is performedafter the image analysis information is set in the computer 31.

This version-up acts as a trigger for causing the computer 31 to resetthe image analysis information on the image data using theversion-updated image analysis engine.

In this example, the trigger detecting unit 110 detects the version-upof the image analysis engine of the computer 31, and the version-up actsas a trigger for causing the setting necessity determining unit 170 todetermine whether or not setting of image analysis information isnecessary.

FIG. 7 illustrates a third example of the case in which an update ofimage analysis information is necessary according to an embodiment ofthe present invention. In this example, it is assumed that each of thecomputer 31 and the server 10 has an image analysis engine of the sameversion. For example, an image analysis engine of the latest version isprovided from another server (not shown) to each of the computer 31 andthe server 10. However, it is also assumed that the precision parameterof the image analysis engine of the computer 31 is lower than that ofthe image analysis engine of the server 10.

In this example, it is assumed that after the image analysis informationis set in the image analysis engine of the computer 31, the computer 31transfers the image analysis information and corresponding image dataand analysis engine information to the server. Since the precisionparameter corresponding to the transferred image analysis information islower than the precision parameter of the image analysis engine of theserver 10, the server 10 performs resetting of image analysisinformation on the image data.

In this example, the trigger detecting unit 110 detects the transfer ofthe image data from the computer 31, and this transfer acts as a triggerfor causing the setting necessity determining unit 170 to determinewhether or not setting of image analysis information is necessary.

FIGS. 8A to 8C illustrate examples of relationships between image dataand image analysis information according to an embodiment of the presentinvention. In the foregoing description, image data, image analysisinformation, and analysis engine information are assumed to be containedin one image file. For example, as illustrated in FIG. 8A, imageanalysis information and analysis engine information corresponding toeach image data are provided as metadata in one image file. Such a fileconfiguration advantageously increases independence of each image fileand facilitates management of individual image files.

Various correspondence relationships between image data and imageanalysis information are possible. For example, as illustrated in FIG.8B, image analysis information and analysis engine information areprovided independently as an image information file, and an image filecontains only image data. With this arrangement, image data, and a pairof image analysis information and analysis engine information can bestored in different recording media. In this case, it is necessary tolink the image data to the pair of image analysis information andanalysis engine information using a common identifier.

Note that there is not necessarily a one-to-one correspondence betweenimage data and analysis engine information. For example, as illustratedin FIG. 8C, it is also possible a piece of analysis engine informationis provided for a plurality of pieces of image data and image analysisinformation. With this arrangement, it is no longer necessary to updateanalysis engine information which is otherwise contained in all imagefiles every time a corresponding analysis engine is updated. Note thatin this case it is necessary to link a pair of image data and imageanalysis information to corresponding analysis engine information usinga common identifier.

FIGS. 9A and 9B are schematic views illustrating an operation part ofthe digital still camera 21. As illustrated in FIG. 9A, when the digitalstill camera 21 is connected to the server 10 through an interface 25,it is determined whether or not image analysis using an image analysisengine of the server 10 is necessary, as described with reference toFIG. 4 and FIG. 5.

If it is determined that setting of image analysis information isnecessary, a confirmation message “Do you wish to update image analysisinformation?” is displayed on a display 26 of the digital still camera21. In response to the message, a user operates a select button 27 toenable or disable the update. When the update is enabled, the imageanalysis information is written back from the server 10 to the digitalstill camera 21.

The write back of image analysis information can also be performedautomatically, without displaying such a confirmation message.

In the following, an operation of an image management apparatusaccording to an embodiment of the present invention will be described.

FIG. 10 illustrates an example of a processing procedure performed by animage management apparatus which corresponds to the server 10 of FIG. 5.In this example, an apparatus corresponding to the digital still camera21 of FIG. 5 is referred to as a “terminal”.

When the trigger detecting unit 110 detects a connection to the terminal(STEP S911), the image management apparatus receives the input imagefile 90 from the terminal, and the data setting unit 181 saves thereceived file as the image data 191 of the saved image file 190 (STEPS912).

If the setting presence determining unit 120 determines the imageanalysis information 92 has already been set (YES, in STEP S913), theprocedure proceeds to processing of updating the image analysisinformation 92 (STEP S930).

On the other hand, if the setting presence determining unit 120determines that the image analysis information 92 has not been set, thefollowing processing is performed. Specifically, the image analysisengine 130 performs image analysis on the image data 91 (STEP S914). Asa result, the analysis information setting unit 182 sets the imageanalysis information 192 of the saved image file 190 (STEP S915). Inaddition, the engine information acquiring unit 150 acquires theanalysis engine information 132 of the image analysis engine 130 (STEPS916), and the engine information setting unit 183 sets the acquiredanalysis engine information 132 as the analysis engine information 193of the saved image file 190 (STEP S917).

The saved image file 190 generated through the above procedure isrecorded in the image database 19 (STEP S918). The image analysisinformation obtained by the image analysis engine 130 and the analysisengine information 132 of the image analysis engine 130 are written backto the terminal (STEP S919).

After the terminal is disconnected from the image management apparatus,the trigger detecting unit 110 waits for another connection (STEP S921).

FIG. 11 illustrates an example of a procedure of image analysisinformation update processing which corresponds to the processing ofSTEP S930 of FIG. 10 according to an embodiment of the presentinvention.

Firstly, the input information acquiring unit 140 acquires the analysisengine information 93 of the input image file 90 (STEP S931). Then, theengine information acquiring unit 150 acquires the analysis engineinformation 132 of the image analysis engine 130 (STEP S932).

The update necessity determining unit 160 compares the versioninformation contained in the analysis engine information 93 with theversion information contained in the analysis engine information 132. Asa result, if the version contained in the analysis engine information 93is earlier than the version contained in the analysis engine information132 (YES, in STEP S933), the following procedure is performed.Specifically, the image analysis engine 130 performs image analysis onthe image data 91 (STEP S936). AS a result, the analysis informationsetting unit 182 resets the image analysis information 192 of the savedimage file 190 (STEP S937). In addition, the engine information settingunit 183 sets the analysis engine information 132 of the image analysisengine 130, which is acquired by the engine information acquiring unit150, as the analysis engine information 193 of the saved image file 190(STEP S938).

If the version contained in the analysis engine information 93 is thesame as the version contained in the analysis engine information 132(YES, in STEP S934), the parameter information contained in the analysisengine information 93 and the parameter information contained in theanalysis engine information 132 are compared. As a result, if theprecision parameter of the analysis engine information 93 is lower thanthat of the analysis engine information 132 (YES, in STEP S935), theprocessing of STEP S936 to STEP 938 described above is performed.

If the version indicated in the analysis engine information 93 is laterthan that indicated in the analysis engine information 132, or if bothversions are the same and the precision parameter contained in theanalysis engine information 93 is not lower than the precision parametercontained in the analysis engine information 132, the analysisinformation setting unit 182 sets the image analysis information 92 asthe image analysis information 192. In addition, the engine informationsetting unit 183 sets the analysis engine information 93 as the analysisengine information 193. That is, no update of image analysis informationis performed.

As described above, according to an embodiment of the present invention,the update necessity determining unit 160 determines whether or not anupdate of image analysis information is necessary on the basis of theanalysis engine information 93 of the input image file 90 and theanalysis engine information 132 of the image analysis engine 130 that isavailable.

Thus, even when version-up of the image analysis engine 130 of theserver 10 is performed, a user can use the updated image analysis engine130 without updating firmware of his or her terminal such as the digitalstill camera 21.

Even in a case where a user purchases a server with increasedperformance, the user can use a result of image analysis performed bythe new server with increased analysis precision by connecting aconventional terminal to the server.

In addition, by including analysis engine information in metadata for animage file, it can be recognized by which image analysis engine themetadata is set and under what parameter the metadata is set. Thus, evenwhen an image file is sent and received between different apparatuses,the same search result can be obtained in all of the apparatuses.

Further, even in the case where an image analysis is composed of aplurality of modules such as face detection and similar image analysis,reset is necessary for only a module that has been updated. This reducesprocessing time and cost.

It should be understood that the above embodiments of the presentinvention illustrate examples for implementing the present invention.The examples illustrated in the embodiments correspond to elements inthe claims. However, the embodiments of the present invention are notlimited to these examples and various modifications may be made withoutdeparting from the scope of the present invention.

Specifically, according to an aspect of the present invention, imageanalyzing means corresponds, for example, to the image analysis engine130, image analysis processing type acquiring means corresponds, forexample, to the engine information acquiring unit 150, and recordingcontrolling means corresponds, for example, to the data setting unit181, the analysis information setting unit 182, and the engineinformation setting unit 183.

According to an aspect of the present invention, input image settinginformation acquiring means corresponds, for example, to the inputinformation acquiring unit 140, available setting information acquiringmeans corresponds, for example, to the engine information acquiring unit150, update necessity determining means corresponds, for example, to theupdate necessity determining unit 160, and image analysis informationsetting means corresponds, for example, to the analysis informationsetting unit 182.

According to an aspect of the present invention, image recording meanscorresponds, for example, to the image recording unit of the computer31, input image setting information acquiring means corresponds, forexample, to the input information acquiring unit 140, available settinginformation acquiring means corresponds, for example, to the engineinformation acquiring unit 150, update necessity determining meanscorresponds, for example, to the update necessity determining unit 160,and image analysis information setting means corresponds, for example,to the analysis information setting unit 182.

According to an aspect of the present invention, image pickup meanscorresponds, for example, to the image pickup unit of the digital stillcamera 21, input image setting information acquiring means corresponds,for example, to the input information acquiring unit 140, availablesetting information acquiring means corresponds, for example, to theengine information acquiring unit 150, update necessity determiningmeans corresponds, for example, to the update necessity determining unit160, and image analysis information setting means corresponds, forexample, to the analysis information setting unit 182.

According to an aspect of the present invention, input image settinginformation acquiring means corresponds, for example, to the inputinformation acquiring unit 140, available setting information acquiringmeans corresponds, for example, to the engine information acquiring unit150, update necessity determining means corresponds, for example, to theupdate necessity determining unit 160, image analysis informationsetting means corresponds, for example, to the analysis informationsetting unit 182, and image analysis information supplying meanscorresponds, for example, to the write back unit 199.

According to an aspect of the present invention, an image analyzing stepcorresponds, for example, to STEP S914, an image analysis processingtype acquiring step corresponds, for example, to STEP S916, and arecording controlling step corresponds, for example, to STEP S918.

According to an aspect of the present invention, an input image settinginformation acquiring step corresponds, for example, to STEP S931, anavailable setting information acquiring step corresponds, for example,to STEP S932, an update necessity determining step corresponds, forexample, to STEP S933 to STEP S935, and an image analysis informationsetting step corresponds, for example, to STEP S936 to STEP S938.

The processing steps described in the above embodiments can beconsidered as a method including a series of processing steps or as aprogram for causing a computer to execute the series of processing stepsor a recording medium for storing the program.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An image recording apparatus comprising: image analyzing meansconfigured to execute predetermined image analysis processing on aninput image so as to extract image analysis information; image analysisprocessing type acquiring means configured to acquire a type of theexecuted image analysis processing, wherein the type of the executedimage analysis processing includes parameter information indicating avalue indicating analysis precision of the image analysis processing forthe image recording apparatus, the parameter information being providedto the image analyzing means and used by the image analyzing means,based on a result of a comparison between the value indicating analysisprecision of the image analysis processing for the image recordingapparatus and a value indicating analysis precision indicated in imageanalysis information associated with the input image, to execute theimage analysis processing; and recording controlling means configured torecord the extracted image analysis information and the type of theimage analysis processing in a recording medium so as to be associatedwith the input image.
 2. An image recording method comprising: executingpredetermined image analysis processing on an input image so as toextract image analysis information; acquiring a type of the executedimage analysis processing, wherein the type of the executed imageanalysis processing includes parameter information indicating a valueindicating analysis precision of the image analysis processing, whereinthe parameter information is provided and used, based on a result of acomparison between the value indicating analysis precision of the imageanalysis processing and a value indicating analysis precision indicatedin image analysis information associated with the input image, in theexecuting image analysis processing; and executing recording control forrecording the extracted image analysis information and the type of theimage analysis processing in a recording medium so as to be associatedwith the input image.
 3. An image recording apparatus comprising: animage analyzing unit configured to execute predetermined image analysisprocessing on an input image so as to extract image analysisinformation; an image analysis processing type acquiring unit configuredto acquire a type of the executed image analysis processing, wherein thetype of the executed image analysis processing includes parameterinformation indicating a value indicating analysis precision of theimage analysis processing for the image recording apparatus, theparameter information being provided to the image analyzing unit andused by the image analyzing unit, based on a result of a comparisonbetween the value indicating analysis precision of the image analysisprocessing for the image recording apparatus and a value indicatinganalysis precision indicated in image analysis information associatedwith the input image, to execute the image analysis processing; and arecording controlling unit configured to record the extracted imageanalysis information and the type of the image analysis processing in arecording medium so as to be associated with the input image.